Data Manipulation & Visualization

class: center, middle, inverse, title-slide

.title[
# Data Manipulation & Visualization
]
.subtitle[
## An Introduction to the {tidyverse} with {refugees} & {unhcthemes}
]
.date[
### <a href="index.html"><i class="fa fa-indent  fa-fw fa-2x"></i></a>
]

---

# Learning objectives

- An intro to Rstudio Development Interface

- Basic of the grammar of data manipulation with [**`dplyr`**](https://dplyr.tidyverse.org/) & [**`tidyr`**](https://tidyr.tidyverse.org/) using [**`{refugees}`**](https://www.unhcr.org/refugee-statistics/insights/explainers/refugees-r-package.html) package

- Basic of the grammar of graphics with [**`ggplot2`**](https://ggplot2.tidyverse.org/index.html)

- Get branded visuals with [**`{unhcrthemes}`**](https://unhcr-dataviz.github.io/unhcrthemes/) package

---

# Learning stages...

.pull-left[

![](data:image/png;base64,#img/learning.png)
]

.pull-right[

__Step 1.__ Develop an understanding of what data science is and what concepts are needed for it
 
 __Step 2.__ Break data science challenges into small steps - Acquire basic command syntax through very practical and focused project - Data Manipulation & visualization
 
 __Step 3.__ Develop Reproducible Analysis Workflow with function and report template  - Understand the relevance, inputs, constraints, and limitations of the various techniques
 
 __Step 4.__ Optimize your problem solving approaches in elegant ways - Build packages & ShinyApp

]

???
https://towardsdatascience.com/the-stages-of-learning-data-science-3cc8be181f54

See Video - https://www.youtube.com/watch?v=hpMc6TgT34I

---
class: inverse, center, middle

# Using Rstudio

---

## Integrated Development Interface (IDE)

.pull-left[
  Environment
  History
  Files
  Plots
  Packages
  Help
  Source
  Console/Terminal
]
.pull-right[
![tidyverse](data:image/png;base64,#img/forcats.png)  
]

---

## Rstudio - Source

.pull-left[
Where you write your R code or document content
If you are writing a R Markdown document, you can render it in this area 
To run a R code, you can use the shortcut Crtl + Enter
]
.pull-right[
![tidyverse](data:image/png;base64,#img/forcats.png)  
]

---

## Rstudio – Console/Terminal

.pull-left[
Where you can check the execution of commands and where the code is evaluated
Can perform quick calculations (that you do not need to save)
]
.pull-right[
![tidyverse](data:image/png;base64,#img/forcats.png)  
]

---

## Rstudio - Environment/History

.pull-left[
Where you can see the objects in your working space 
You can also view your command history (History tab) 
]
.pull-right[
![tidyverse](data:image/png;base64,#img/forcats.png)  
]

---

## Rstudio – Files/Plots/Packages/Help

.pull-left[
You can see the file directories
View plots
View which R packages are installed. You can also update the packages
]
.pull-right[
![tidyverse](data:image/png;base64,#img/forcats.png)  
]
R help

---
class: inverse, center, middle

# Manipulate data on {refugees} with {dplyr} & {tidyr}

---

### Install the {tidyverse}, a collection of opinionated packages!

Got to your [locally installed Rstudio](https://www.rstudio.com/products/rstudio/download/#download) or [sign-up for a free Rstudio Cloud account](https://login.rstudio.cloud/register?redirect=https%3A%2F%2Fclient.login.rstudio.cloud%2Foauth%2Flogin%3Fshow_auth%3D0%26show_login%3D0)

.pull-left[

First create a new project within R studio and then make sure we have the [tidyverse](https://www.tidyverse.org/packages/)

```r
# Tidyverse
if (!require("tidyverse")) 
  install.packages("tidyverse", 
                   dependencies = TRUE)
if (!require("here")) 
  install.packages("here")
```
]

.pull-right[

![tidyverse](data:image/png;base64,#img/forcats.png)

]

???

The tidyverse is an opinionated collection of R packages designed for data science. All packages share an underlying design philosophy, grammar, and data structures.

---

## The {refugees} Data Package

A package can be made of both functions and data.

{refugees} is easy-to-use interface to the datasets served normally through the API, which cover forcibly displaced populations, including refugees, asylum-seekers and internally displaced people, stateless people, and others over a span of more than 70 years. This package provides data from three major sources:

.pull-left[

Data from UNHCR’s annual statistical activities dating back to 1951.
 
Data from the United Nations Relief and Works Agency for Palestine Refugees in the Near East (UNRWA), specifically for registered Palestine refugees under UNRWA’s mandate. 
 
Data from the Internal Displacement Monitoring Centre (IDMC) on people displaced within their country due to conflict or violence.

]

.pull-right[

```r

## 2 options to install

# From CRAN
# install.packages("refugees")

# From Github
# pak::pkg_install("PopulationStatistics/refugees")

## Once installed, load the package!
library(refugees)
```
]

---

## 8 sub datasets in the package - update every 6 months

1. `population`: Data on forcibly displaced and stateless persons by year, including refugees, asylum-seekers, internally displaced people (IDPs) and stateless people. Detailed definitions of the different population groups can be found on the methodology page of the Refugee Data Finder.
 
 2. `idmc`: Data from the Internal Displacement Monitoring Centre on the total number of IDPs displaced due to conflict and violence.
 
 3. `asylum_applications`: Data on asylum applications including the procedure type and application type.
 
 4. `asylum_decisions`: Data on asylum decisions, including recognitions, rejections, and administrative closures.
 
 5.`demographics`: Demographic and sub-national data, where available, including disaggregation by age and sex.

6. `solutions`: Data on durable solutions for refugees and IDPs.
  
 7. `unrwa`: Data on registered Palestine refugees under UNRWA’s mandate.
 
 8. `flows`: Numbers of the people forced to flee during each of the years since 1962. For more information, see the explaination of the forced displacement flow dataset.

---

## Data Manipulation

filter, group_by, summarise, slice

```r
library(refugees)
library(dplyr)
library(tidyr)

ref_coo_10 <- refugees::population |>
  dplyr::filter(year == 2022) |>
  dplyr::group_by(coo_name) |>
  dplyr::summarise(refugees = sum(refugees, na.rm = TRUE) + sum(oip, na.rm = TRUE) ) |>
  dplyr::slice_max(order_by = refugees, n = 10)
```

---

## Data Manipulation

left_join

```r
fd_last_ten_years <- refugees::population |>
  dplyr::filter(year >= 2022-9 & year < 2023) |>
  dplyr::summarise(refugees = sum(refugees, na.rm = TRUE),
            asylum_seekers = sum(asylum_seekers, na.rm = TRUE), 
            oip = sum(oip, na.rm = TRUE), .by = year) |>
  dplyr::left_join(refugees::idmc |> 
              filter(year >= 2022-9) |>
              summarise(idmc = sum(total, na.rm = TRUE),
                        .by = year),
            by=c("year")) |>
  dplyr::left_join(refugees::unrwa |> 
              filter(year >= 2022-9) |>
              summarise(unrwa = sum(total, na.rm = TRUE), .by = year), by=c("year"))  
```

---

## Data Manipulation

pivot_longer

```r
fd_last_ten_years <- fd_last_ten_years |>
  tidyr::pivot_longer(cols = -year, 
               names_to = "population_type", 
               values_to = "total") |> 
  dplyr::mutate(population_type = factor(population_type,
                                          levels=c("oip", "unrwa","asylum_seekers", "refugees","idmc")), 
             population_type = dplyr::recode(population_type,
             refugees="Refugees under UNHCR’s mandate",
             asylum_seekers="Asylum-seekers",
             oip="Other people in need of international protection",
             idmc="Internally displaced persons",
             unrwa="Palestine refugees under UNRWA’s mandate")) |>
  dplyr::arrange(year, population_type)
```

---

## Data Manipulation

sum with na.rm

```r
demo_2022 <- refugees::demographics |> 
  filter(year == 2022 & 
         pop_type %in% c("REF", "ASY", "IDP", "OIP",
                         "RDP", "RET", "STA", "OOC")) |>
  summarise("male 0-17" = sum(m_0_4, na.rm = TRUE) + 
                          sum(m_5_11, na.rm = TRUE) + 
                          sum(m_12_17, na.rm = TRUE),
            "male 18-59"= sum(m_18_59, na.rm = TRUE), 
            "male 60+"=sum(m_60, na.rm=TRUE), 
            "female 0-17" = sum(f_0_4, na.rm = TRUE) + 
                            sum(f_5_11, na.rm = TRUE) + 
                            sum(f_12_17, na.rm = TRUE),
            "female 18-59"= sum(f_18_59, na.rm = TRUE), 
            "female 60+"=sum(f_60, na.rm=TRUE)) |>
  pivot_longer(cols=everything(), 
               names_sep = " ", 
               names_to = c(".value", "ages")) |>
  mutate(male_p = round(male/(sum(female) + sum(male)), 2 ),   
         female_p = round(female/(sum(female) + sum(male)), 2 )) 
  
```

---
class: inverse, center, middle

# Visualise with {ggplot2} and {unhcrthemes}

### Introduction

---

## The `ggplot2` package

.pull-left[

- **`ggplot2`** is an R package for declaratively creating graphics

- **`ggplot2`** is an implementation [The Grammar of Graphics](https://link.springer.com/chapter/10.1007/978-3-642-21551-3_13) by Leland Irving

- **The idea** don't start with the final form of the graphic (Excel approach) but **decompose the graphic** into its constituents

]

.pull-right[

.center[![ggplot2 HEX](data:image/png;base64,#https://raw.githubusercontent.com/tidyverse/ggplot2/main/man/figures/logo.png)]

]

???
You provide the data, tell ggplot2 how to map variables to aesthetics, what graphical primitives to use, and it takes care of the details.
What does it take to create a graphic? Data, axis, geometric objects, etc.

---

## Structure of `ggplot2`